Comparing extinction and contrast effects for operant wheel running and lever pressing on a multiple schedule.

Extinction and positive contrast effects were assessed on a multiple schedule with lever pressing as the operant in one component and wheel running as the operant in the other component. FR 15 schedules produced 15% sucrose reinforcement in each component. Contrast for both operants was generated by placing responding in the alternate component on extinction. Results showed that extinction decreased and contrast increased both lever-pressing and wheel-running rates. However, the magnitude of the changes was greater for lever pressing. Extinction increased and contrast decreased postreinforcement pause (PRP) duration for lever pressing, but for wheel running, extinction decreased PRP duration while contrast did not change PRP duration. Finally, outcomes for lever pressing decreased with extinction and increased with contrast, but for wheel running, outcomes did not change with extinction and increased with contrast. These differences in contrast and extinction effects were explained by an automatic reinforcement effect generated by wheel-running, but not lever pressing. These findings provide further support for an automatic reinforcement effect generated by wheel running.

Effects of pre-operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement.

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.

Effect of amphetamine dose on wheel-running functioning as reinforcement or operant behavior on a multiple schedule of reinforcement.

Does the effect of amphetamine on behavior (wheel running) differ depending on the functional role (operant, reinforcement) of that behavior? This study addressed this question using a multiple schedule of reinforcement in which wheel running served as reinforcement for lever pressing in one component and as operant behavior for sucrose reinforcement in the other component. Seven female Long-Evans rats were exposed to a multiple schedule in which pressing a lever on a variable ratio 10 schedule produced the opportunity to run for 15 revolutions in one component and running 15 revolutions produced a drop of 15% sucrose solution in the other component. Doses of 0.5, 1.0, and 2.0 mg/kg D-amphetamine were administered by intraperitoneal injection 20 min prior to a session. As amphetamine dose increased, wheel running decreased in both components - showing no evidence that the effect of the drug on wheel running depended on the function of wheel activity. Notably, lever pressing for wheel-running reinforcement also decreased with amphetamine dose. Drug dose increased the initiation of operant lever pressing, but not the initiation of operant wheel running. We propose that amphetamine dose had common effects on wheel running regardless of its function (reinforcement vs. operant) because wheel-running generates automatic reinforcement and the automatic-reinforcement value of wheel activity is modulated by drug dose.

Automatic reinforcement from operant wheel-running undermines temporal control by fixed-interval schedules of reinforcement.

The current study compared the development of response patterns for operant wheel-running and lever-pressing on fixed-interval schedules. Eleven female Long-Evans rats were exposed to fixed-interval (FI) 15-s, 30-s, and 60-s schedules with wheel revolutions as the operant behavior and sucrose solution as reinforcement. Subsequently, a lever was mounted in each wheel and rats responded on an FI-30 s schedule of sucrose reinforcement. Operant lever-pressing on average developed a scalloping pattern of low responding early in the reinforcement interval followed by an increase in pressing to the moment of reinforcement. In contrast, average operant wheel-revolutions peaked early in the reinforcement interval followed by a plateau, a pattern that did not change over sessions. Variation in the FI-schedule value (interval size) with operant wheel-running did not alter the pattern of running throughout the reinforcement interval, but merely parsed this pattern at different points. Cumulative records for the last session showed long postreinforcement pauses (PRP) for lever pressing. Wheel running, however, rose quickly after reinforcement and continued throughout the reinforcement interval. Overall and local wheel-running rates decreased and PRP duration increased as the interval size of the FI schedule increased. We propose that the automatic reinforcement generated by wheel running, but not lever pressing, provides an account of the poor temporal regulation of operant wheel-running in our study.

Evidence for positive, but not negative, behavioral contrast with wheel-running reinforcement on multiple variable-ratio schedules.

Rats responded on a multiple variable-ratio (VR) 10 VR 10 schedule of reinforcement in which lever pressing was reinforced by the opportunity to run in a wheel for 30s in both the changed (manipulated) and unchanged components. To generate positive contrast, the schedule of reinforcement in the changed component was shifted to extinction; to generate negative contrast, the schedule was shifted to VR 3. With the shift to extinction in the changed component, wheel-running and local lever-pressing rates increased in the unchanged component, a result supporting positive contrast; however, the shift to a VR 3 schedule in the changed component showed no evidence of negative contrast in the unaltered setting, only wheel running decreased in the unchanged component. Changes in wheel-running rates across components were consistent in showing a compensation effect, depending on whether the schedule manipulation increased or decreased opportunities for wheel running in the changed component. These findings are the first to demonstrate positive behavioral contrast on a multiple schedule with wheel running as reinforcement in both components.

Wheel-running reinforcement in free-feeding and food-deprived rats.

Rats experiencing sessions of 30min free access to wheel running were assigned to ad-lib and food-deprived groups, and given additional sessions of free wheel activity. Subsequently, both ad-lib and deprived rats lever pressed for 60s of wheel running on fixed ratio (FR) 1, variable ratio (VR) 3, VR 5, and VR 10 schedules, and on a response-initiated variable interval (VI) 30s schedule. Finally, the ad-lib rats were switched to food deprivation and the food-deprived rats were switched to free food, as rats continued responding on the response-initiated VI 30-s schedule. Wheel running functioned as reinforcement for both ad-lib and food-deprived rats. Food-deprived rats, however, ran faster and had higher overall lever-pressing rates than free-feeding rats. On the VR schedules, wheel-running rates positively correlated with local and overall lever pressing rates for deprived, but not ad-lib rats. On the response-initiated VI 30s schedule, wheel-running rates and lever-pressing rates changed for ad-lib rats switched to food deprivation, but not for food-deprived rats switched to free-feeding. The overall pattern of results suggested different sources of control for wheel running: intrinsic motivation, contingencies of automatic reinforcement, and food-restricted wheel running. An implication is that generalizations about operant responding for wheel running in food-deprived rats may not extend to wheel running and operant responding of free-feeding animals.

Effect of sucrose availability on wheel-running as an operant and as a reinforcing consequence on a multiple schedule: Additive effects of extrinsic and automatic reinforcement.

As a follow up to Belke and Pierce's (2014) study, we assessed the effects of repeated presentation and removal of sucrose solution on the behavior of rats responding on a two-component multiple schedule. Rats completed 15 wheel turns (FR 15) for either 15% or 0% sucrose solution in the manipulated component and lever pressed 10 times on average (VR 10) for an opportunity to complete 15 wheel turns (FR 15) in the other component. In contrast to our earlier study, the components advanced based on time (every 8min) rather than completed responses. Results showed that in the manipulated component wheel-running rates were higher and the latency to initiate running longer when sucrose was present (15%) compared to absent (0% or water); the number of obtained outcomes (sucrose/water), however, did not differ with the presentation and withdrawal of sucrose. For the wheel-running as reinforcement component, rates of wheel turns, overall lever-pressing rates, and obtained wheel-running reinforcements were higher, and postreinforcement pauses shorter, when sucrose was present (15%) than absent (0%) in manipulated component. Overall, our findings suggest that wheel-running rate regardless of its function (operant or reinforcement) is maintained by automatically generated consequences (automatic reinforcement) and is increased as an operant by adding experimentally arranged sucrose reinforcement (extrinsic reinforcement). This additive effect on operant wheel-running generalizes through induction or arousal to the wheel-running as reinforcement component, increasing the rate of responding for opportunities to run and the rate of wheel-running per opportunity.

Effect of sucrose availability and pre-running on the intrinsic value of wheel running as an operant and a reinforcing consequence.

The current study investigated the effect of motivational manipulations on operant wheel running for sucrose reinforcement and on wheel running as a behavioral consequence for lever pressing, within the same experimental context. Specifically, rats responded on a two-component multiple schedule of reinforcement in which lever pressing produced the opportunity to run in a wheel in one component of the schedule (reinforcer component) and wheel running produced the opportunity to consume sucrose solution in the other component (operant component). Motivational manipulations involved removal of sucrose contingent on wheel running and providing 1h of pre-session wheel running. Results showed that, in opposition to a response strengthening view, sucrose did not maintain operant wheel running. The motivational operations of withdrawing sucrose or providing pre-session wheel running, however, resulted in different wheel-running rates in the operant and reinforcer components of the multiple schedule; this rate discrepancy revealed the extrinsic reinforcing effects of sucrose on operant wheel running, but also indicated the intrinsic reinforcement value of wheel running across components. Differences in wheel-running rates between components were discussed in terms of arousal, undermining of intrinsic motivation, and behavioral contrast.

Contingency discriminability and the generalized matching law describe choice on concurrent ratio schedules of wheel-running reinforcement.

Belke (2010) showed that on concurrent ratio schedules, the difference in ratio requirements required to produce near exclusive preference for the lower ratio alternative was substantively greater when the reinforcer was wheel running than when it was sucrose. The current study replicated this finding and showed that this choice behavior can be described by the matching law and the contingency discriminability model. Eight female Long Evans rats were exposed to concurrent VR schedules of wheel-running reinforcement (30s) and the schedule value of the initially preferred alternative was systematically increased. Two rats rapidly developed exclusive preference for the lower ratio alternative, but the majority did not - even when ratios differed by 20:1. Analysis showed that estimates of slopes from the matching law and the proportion of reinforcers misattributed from the contingency discriminability model were related to the ratios at which near exclusive preference developed. The fit of these models would be consistent with misattribution of reinforcers or poor discrimination between alternatives due to the long duration of wheel running.

Exclusive preference develops less readily on concurrent ratio schedules with wheel-running than with sucrose reinforcement.

Previous research suggested that allocation of responses on concurrent schedules of wheel-running reinforcement was less sensitive to schedule differences than typically observed with more conventional reinforcers. To assess this possibility, 16 female Long Evans rats were exposed to concurrent FR FR schedules of reinforcement and the schedule value on one alternative was systematically increased. In one condition, the reinforcer on both alternatives was .1 ml of 7.5% sucrose solution; in the other, it was a 30-s opportunity to run in a wheel. Results showed that the average ratio at which greater than 90% of responses were allocated to the unchanged alternative was higher with wheel-running reinforcement. As the ratio requirement was initially increased, responding strongly shifted toward the unchanged alternative with sucrose, but not with wheel running. Instead, responding initially increased on both alternatives, then subsequently shifted toward the unchanged alternative. Furthermore, changeover responses as a percentage of total responses decreased with sucrose, but not wheel-running reinforcement. Finally, for some animals, responding on the increasing ratio alternative decreased as the ratio requirement increased, but then stopped and did not decline with further increments. The implications of these results for theories of choice are discussed.

Body weight manipulation, reinforcement value and choice between sucrose and wheel running: a behavioral economic analysis.

Twelve female Long-Evans rats were exposed to concurrent variable (VR) ratio schedules of sucrose and wheel-running reinforcement (Sucrose VR 10 Wheel VR 10; Sucrose VR 5 Wheel VR 20; Sucrose VR 20 Wheel VR 5) with predetermined budgets (number of responses). The allocation of lever pressing to the sucrose and wheel-running alternatives was assessed at high and low body weights. Results showed that wheel-running rate and lever-pressing rates for sucrose and wheel running increased, but the choice of wheel running decreased at the low body weight. A regression analysis of relative consumption as a function of relative price showed that consumption shifted toward sucrose and interacted with price differences in a manner consistent with increased substitutability. Demand curves showed that demand for sucrose became less elastic while demand for wheel running became more elastic at the low body weight. These findings reflect an increase in the difference in relative value of sucrose and wheel running as body weight decreased. Discussion focuses on the limitations of response rates as measures of reinforcement value. In addition, we address the commonalities between matching and demand curve equations for the analysis of changes in relative reinforcement value.

A brief opportunity to run does not function as a reinforcer for mice selected for high daily wheel-running rates.

Mice from replicate lines, selectively bred based on high daily wheel-running rates, run more total revolutions and at higher average speeds than do mice from nonselected control lines. Based on this difference it was assumed that selected mice would find the opportunity to run in a wheel a more efficacious consequence. To assess this assumption within an operant paradigm, mice must be trained to make a response to produce the opportunity to run as a consequence. In the present study an autoshaping procedure was used to compare the acquisition of lever pressing reinforced by the opportunity to run for a brief opportunity (i.e., 90 s) between selected and control mice and then, using an operant procedure, the effect of the duration of the opportunity to run on lever pressing was assessed by varying reinforcer duration over values of 90 s, 30 min, and 90 s. The reinforcement schedule was a ratio schedule (FR 1 or VR 3). Results from the autoshaping phase showed that more control mice met a criterion of responses on 50% of trials. During the operant phase, when reinforcer duration was 90 s, almost all control, but few selected mice completed a session of 20 reinforcers; however, when reinforcer duration was increased to 30 min almost all selected and control mice completed a session of 20 reinforcers. Taken together, these results suggest that selective breeding based on wheel-running rates over 24 hr may have altered the motivational system in a way that reduces the reinforcing value of shorter running durations. The implications of this finding for these mice as a model for attention deficit hyperactivity disorder (ADHD) are discussed. It also is proposed that there may be an inherent trade-off in the motivational system for activities of short versus long duration.

Rats do not respond differently in the presence of stimuli signaling wheel-running reinforcers of different durations.

Rats were exposed to a fixed interval 30 s schedule that produced opportunities to run of equal or unequal durations to assess the effect of differences in duration on responding. Each duration was signaled by a different stimulus. Wheel-running reinforcer duration pairs were 30 s 30 s, 50 s 10 s, and 55 s 5 s. An analysis of median postreinforcement pause duration and mean local lever-pressing rates broken down by previous reinforcer duration and duration of signaled upcoming reinforcer showed that postreinforcement pause duration was affected by the duration of the previous reinforcer but not by the stimulus signaling the duration of the upcoming reinforcer. Local lever-pressing rates were not affected by either previous or upcoming reinforcer duration. In general, the results are consistent with indifference between these durations obtained using a concurrent choice procedure.

Reinforcement value and substitutability of sucrose and wheel running: implications for activity anorexia.

Choice between sucrose and wheel-running reinforcement was assessed in two experiments. In the first experiment, ten male Wistar rats were exposed to concurrent VI 30 s VI 30 s schedules of wheel-running and sucrose reinforcement. Sucrose concentration varied across concentrations of 2.5, 7.5, and 12.5%. As concentration increased, more behavior was allocated to sucrose and more reinforcements were obtained from that alternative. Allocation of behavior to wheel running decreased, but obtained wheel-running reinforcement did not change. Overall, the results suggested that food-deprived rats were sensitive to qualitative changes in food supply (sucrose concentration) while continuing to defend a level of physical activity (wheel running). In the second study, 15 female Long Evans rats were exposed to concurrent variable ratio schedules of sucrose and wheel-running, wheel-running and wheel-running, and sucrose and sucrose reinforcement. For each pair of reinforcers, substitutability was assessed by the effect of income-compensated price changes on consumption of the two reinforcers. Results showed that, as expected, sucrose substituted for sucrose and wheel running substituted for wheel running. Wheel running, however, did not substitute for sucrose; but sucrose partially substituted for wheel running. We address the implications of the interrelationships of sucrose and wheel running for an understanding of activity anorexia.

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement.

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.

Concurrent schedules of wheel-running reinforcement: choice between different durations of opportunity to run in rats.

How do animals choose between opportunities to run of different durations? Are longer durations preferred over shorter durations because they permit a greater number of revolutions? Are shorter durations preferred because they engender higher rates of running? Will longer durations be chosen because running is less constrained? The present study reports on three experiments that attempted to address these questions. In the first experiment, five male Wistar rats chose between 10-sec and 50-sec opportunities to run on modified concurrent variable-interval (VI) schedules. Across conditions, the durations associated with the alternatives were reversed. Response, time, and reinforcer proportions did not vary from indifference. In a second experiment, eight female Long-Evans rats chose between opportunities to run of equal (30 sec) and unequal durations (10 sec and 50 sec) on concurrent variable-ratio (VR) schedules. As in Experiment 1, between presentations of equal duration conditions, 10-sec and 50-sec durations were reversed. Results showed that response, time, and reinforcer proportions on an alternative did not vary with reinforcer duration. In a third experiment, using concurrent VR schedules, durations were systematically varied to decrease the shorter duration toward 0 sec. As the shorter duration decreased, response, time, and reinforcer proportions shifted toward the longer duration. In summary, differences in durations of opportunities to run did not affect choice behavior in a manner consistent with the assumption that a longer reinforcer is a larger reinforcer.

Responding for sucrose and wheel-running reinforcement: effect of pre-running.

Six male albino Wistar rats were placed in running wheels and exposed to a fixed interval 30-s schedule that produced either a drop of 15% sucrose solution or the opportunity to run for 15s as reinforcing consequences for lever pressing. Each reinforcer type was signaled by a different stimulus. To assess the effect of pre-running, animals were allowed to run for 1h prior to a session of responding for sucrose and running. Results showed that, after pre-running, response rates in the later segments of the 30-s schedule decreased in the presence of a wheel-running stimulus and increased in the presence of a sucrose stimulus. Wheel-running rates were not affected. Analysis of mean post-reinforcement pauses (PRP) broken down by transitions between successive reinforcers revealed that pre-running lengthened pausing in the presence of the stimulus signaling wheel running and shortened pauses in the presence of the stimulus signaling sucrose. No effect was observed on local response rates. Changes in pausing in the presence of stimuli signaling the two reinforcers were consistent with a decrease in the reinforcing efficacy of wheel running and an increase in the reinforcing efficacy of sucrose. Pre-running decreased motivation to respond for running, but increased motivation to work for food.

Habituation contributes to the decline in wheel running within wheel-running reinforcement periods.

Habituation appears to play a role in the decline in wheel running within an interval. Aoyama and McSweeney [Aoyama, K., McSweeney, F.K., 2001. Habituation contributes to within-session changes in free wheel running. J. Exp. Anal. Behav. 76, 289-302] showed that when a novel stimulus was presented during a 30-min interval, wheel-running rates following the stimulus increased to levels approximating those earlier in the interval. The present study sought to assess the role of habituation in the decline in running that occurs over a briefer interval. In two experiments, rats responded on fixed-interval 30-s schedules for the opportunity to run for 45 s. Forty reinforcers were completed in each session. In the first experiment, the brake and chamber lights were repeatedly activated and inactivated after 25 s of a reinforcement interval had elapsed to assess the effect on running within the remaining 20 s. Presentations of the brake/light stimulus occurred during nine randomly determined reinforcement intervals in a session. In the second experiment, a 110 dB tone was emitted after 25 s of the reinforcement interval. In both experiments, presentation of the stimulus produced an immediate decline in running that dissipated over sessions. No increase in running following the stimulus was observed in the first experiment until the stimulus-induced decline dissipated. In the second experiment, increases in running were observed following the tone in the first session as well as when data were averaged over several sessions. In general, the results concur with the assertion that habituation plays a role in the decline in wheel running that occurs within both long and short intervals.

The reinforcing property and the rewarding aftereffect of wheel running in rats: a combination of two paradigms.

Wheel running reinforces the behavior that generates it and produces a preference for the context that follows it. The goal of the present study was to demonstrate both of these effects in the same animals. Twelve male Wistar rats were first exposed to a fixed-interval 30 s schedule of wheel-running reinforcement. The operant was lever-pressing and the reinforcer was the opportunity to run for 45 s. Following this phase, the method of place conditioning was used to test for a rewarding aftereffect following operant sessions. On alternating days, half the rats responded for wheel-running reinforcement while the other half remained in their home cage. Upon completion of the wheel-running reinforcement sessions, rats that ran and rats that remained in their home cages were placed into a chamber of a conditioned place preference (CPP) apparatus for 30 min. Each animal received six pairings of a distinctive context with wheel running and six pairings of a different context with their home cage. On the test day, animals were free to move between the chambers for 10 min. Results showed a conditioned place preference for the context associated with wheel running; however, time spent in the context associated with running was not related to wheel-running rate, lever-pressing rate, or post-reinforcement pause duration.

Effect of short-term prefeeding and body weight on wheel running and responding reinforced by the opportunity to run in a wheel.

A biobehavioural analysis of activity anorexia suggests that the motivation for physical activity is regulated by food supply and body weight. In the present experiment, food allocation was varied within subjects by prefeeding food-deprived rats 0, 5, 10 and 15 g of food before sessions of lever pressing for wheel-running reinforcement. The experiment assessed the effects of prefeeding on rates of wheel running, lever pressing, and postreinforcement pausing. Results showed that prefeeding animals 5 g of food had no effect. Prefeeding 10 g of food reduced lever pressing for wheel running and rates of wheel running without a significant change in body weight; the effect was, however, transitory. Prefeeding 15 g of food increased the animals' body weights, resulting in a sustained decrease of wheel running and lever pressing, and an increase in postreinforcement pausing. Overall the results indicate that the motivation for physical activity is regulated by changes in local food supply, but is sustained only when there is a concomitant change in body weight.